An electronic imaging system depends on a lens system to form an image on an image sensor to create an electronic representation of a visual image. Examples of such image sensors include charge coupled device (CCD) image sensors and active pixel sensor (APS) devices (APS devices are often referred to as CMOS sensors because of the ability to fabricate them in a Complementary Metal Oxide Semiconductor process). A sensor includes a two-dimensional array of individual picture element sensors, or pixels. Each pixel is typically provided with either a red, green, or blue filter, as for example described by Bayer in U.S. Pat. No. 3,971,065 issued Jul. 20, 1976, so that a full color image can be produced. Regardless of the type of image sensor employed, e.g., CCD or CMOS, the pixel acts as a bucket in which photo-generated charge is accumulated in direct proportion to the amount of light that strikes the pixel during the capture of an image by the electronic imaging system.
The image sensor gathers light for an interval of time called the exposure time or integration time to make a correct exposure during image capture. Based on brightness measurements of the scene to be imaged, the electronic imaging system, typically with an autoexposure system, is employed to determine a suitable exposure time that will yield an image with effective brightness and an effective signal to noise ratio. The dimmer the scene, the larger the amount of time the electronic imaging system must use to gather light to make a correct exposure. If motion relative to the image capture device is present during image capture, motion blur can be present in the captured image as the motion velocity increases relative to the exposure time. There are two types of motion blur: global motion blur and local motion blur. Global motion blur is produced when the image capture device is moving relative to the scene during capture and as such the entire image is blurred. Methods to reduce global motion blur are well known to those in the field. One method is to use an inertial measurement device (typically a gyro) to measure the motion of the image capture device during capture and then use a special lens with a lens element that can be moved laterally to cause the image formed by the lens on the image sensor to move in a direction that compensates for the image capture device motion. A second method is described in U.S. Patent Publication No. 2006/0274156, based on a digital shifting of images to compensate for movement of the digital camera during video captures to reduce global motion blur and to stabilize the images in the video. However, neither of these methods address local motion blur produced within the scene.
One method to reduce local motion blur is to shorten the exposure time. This method reduces the exposure time to something less than the exposure time selected by the autoexposure system; as a result, darker images with a low signal to noise ratio are captured. An analog or digital gain can then be applied to the pixel values in the image to brighten the darker images, but those skilled in the art will recognize that this will result in noisier images.
Another method to reduce local motion blur is to gather more light with a larger aperture lens and larger pixels on the image sensor, enabling a shorter exposure time. This method can produce images with reduced local motion blur and acceptable noise levels. However, the current industry trend in electronic imaging systems is to make image capture devices more compact and less expensive. High-grade optical elements with large apertures and image sensors with larger pixels, which can gather more light, are therefore not practical.
Another method to reduce local motion blur is to shorten the exposure time and supplement the available light with a photographic flash. A photographic flash produces a strong light flux that is sustained for a fraction of a second with the exposure time set to encompass the flash time. The exposure time can be set to a significantly shorter time interval than without a flash since the photographic flash is very bright. Therefore, the blur caused by local motion during the exposure is reduced. However, flashes are not effective in bright lighting and fast moving objects in bright lighting can still produce local motion blur. In addition, flash photography is typically only useful if the distance between the flash and the scene being photographed is small. Flash photography also tends to produce artifacts such as red eyes and very bright areas or dark areas, which many people find objectionable.
Conventional solutions for selecting exposure time typically use one or more standardized settings, or respond to operator mode settings to obtain an exposure time. FIG. 2A shows a flow chart of a typical camera control system 200 for a digital camera as is typically performed in an autoexposure control system. In Step 210, the camera assesses the scene brightness either with a scene brightness sensor or with an analysis of a preview image. In the typical camera control system shown in FIG. 2A, motion is not measured or taken into account. In Step 220, the capture mode is determined based on the scene brightness and any operator settings or standardized settings. In Step 230, the ISO is determined in accordance with the scene brightness and the capture mode. The exposure time is then determined in Step 240 in accordance with the scene brightness, the capture mode and the ISO. In Step 250, a final archival image is captured and stored. However, the method of the typical camera control system 200 can capture images with poor perceived image quality because the degree of brightness and motion in the scene can be highly variable and since motion is not taken into account, disappointing levels of motion blur or noise can be present in the images.
In US Patent Publication 2007/0237514, motion in the scene is measured prior to image capture. If slow motion is detected, then additional analysis is done to help select a scene mode for the camera. If rapid motion is detected, then a set of standardized camera settings is used as determined by the autoexposure control system and the operator selectable camera settings as presented in FIG. 2A. As such, the method of 2007/0237514 only provides an improved method for capture of scenes with slow motion.
In US Patent Publication 2007/0237506, a camera is described wherein an image is captured at a slower shutter speed if no camera motion is detected. If camera motion is detected, then an image is captured at a faster shutter speed. While this method does reduce motion blur in images, it does not address the combined effects of motion blur and noise in the image on the perceived image quality of the image in selecting capture conditions including exposure time and ISO. The method of 2007/0237506, also does not include the selection of capture conditions based on local motion.
FIG. 2B shows a flow chart 290 of the PRIOR ART described in WO2007017835 for a method of control of a multiple image capture process. In Step 270 a first image is captured using exposure conditions defined by the camera autoexposure control system as presented in FIG. 2A. In Step 275, the first image is then analyzed for aspects of image quality such as overexposure or underexposure, motion blur, dynamic range or depth of field to determine which aspects have been met and where deficiencies remain. Based on this analysis, Step 280 is a check on whether deficiencies remain in the aspects of image quality. If some deficiencies remain in the aspects of image quality, the process proceeds to Step 282 where new exposure parameters are set for at least one additional image. The process then loops back to Step 272 where at least one additional image is captured using the new exposure parameters. The additional image is then analyzed in Step 275 for aspects of image quality. This process repeats until all the aspects of image quality have been met amongst the multiple images that have been captured. A final image is then constructed from portions of the multiple images that are combined in Step 285 in such a way that all the aspects of image quality desired are met. However, the method of WO2007017835 is directed at the control of multiple image capture photography where portions of multiple images are combined to produce an improved image and does not address motion related issues in a single captured image.
Thus, there is a need for an automated method for selecting capture conditions to improve perceived image quality in a single captured image when global motion or local motion is present in the scene being imaged.